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ABSTRACT 

The Jasper Challenge Series of Vanderbilt University 
(Tennessee) has been used in conjunction with the university's 
videodisk problem-solving series, "The Adventures of Jasper 
Woodbury, 11 to provide a teleconference-based performance arena that 
allows students and teachers to assess the degree to which they are 
learning to solve the kinds of problems that the Jasper series 
emphasizes. Students from a number of sites using the Jasper program 
jointly participate in teleconference challenges that allow them to 
choose the characters in the teleconference who do the best job of 
solving problems. They receive feedback and compare their ideas with 
peers from other cities and states. Pilot Satellite Challenges have 
been uplinked to over 30 sites across seven states. These Special 
Multimedia Arenas for Refining Thinking (SMART) challenges are 
described, and the challenge-based assessment model that has 
developed is explored. The first SMART challenge involved over 500 
fifth and sixth graders in a game show called "Pick the Expert." A 
subsequent challenge involved a large group of students in a more 
difficult game show. A third challenge involved a single local 
classroom in a developmental exploration of teaching and learning 
before the challenge. Future plans for the SMART program are 
discussed, and the model that is being developed is reviewed. 
(Contains 15 references.) (SLD) 
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THE SATELLITE CHALLENGE SERIES 



Ronald J. Kantor, Allison L. Moore, and John D. Bransford 
with the Cognition and Technology Group at Vanderbilt 

ABSTRACT 

The impact of classsroom-based technology on learning can be extended through the 
innovative use of distance learning technologies. Vanderbilt University's Jasper Challenge Series 
Is one example of the possibilities that exist. The Challenge Series has been used in conjunction 
with Vanderbilt' s Jasper Woodbury Problem Solving Series. It provides a teleconference-based 
performance arena that allows students and teachers to assess the degree to which they are learning 
to solve the kinds of problems that the Jasper series emphasizes. 

In the Challenges, students from a number of different Jasper sites jointly participate in 
teleconference-based challenges that allow them to "test their mettle" by choosing the characters on 
the teleconference who do the best job of solving some difficult problems. Students need to be 
expert problem solvers in the domain being investigated in order to make the appropriate choices. 
Students call in their votes at the end of the challenge and have the opportunity to compare their 
ideas with those of peers from other cities and states. They then receive feedback about the 
strengths and weaknesses of each choice. Later, they get a chance to participate in new, video- 
based challenge. The Challenge Series also provides a forum for displaying student-generated 
projects that are relevant to the theme of each challenge. 

Discussion will focus on the lessons learned from our first four Challenge Series pilots. 
Teachers' reactions to the challenges will be discussed as well. It will be argued that the general 
approach of the challenge-based assessment model we have developed can be used to enhance 
formative assessment and subsequent learning in any problem or project-based curriculum. Design 
principles for successful Challenges will be enumerated and discussed. 

INTRODUCTION 

The Jasper Satellite Challenge Series is a prototype for a new form of interactive, challenge 
based assessment which is designed to be applicable to any project based curriculum. Initially 
developed within the context of Vanderbilt University's The Adventures of Jasper Woodbury 
problem solving series, a series of videodisc based adventures designed to develop mathematical 
problem solving and critical thinking skills in middle school students through collaborative 
learning, the Challenges provide motivating public performance arenas that allows students to "test 
their mettle" with respect to the Jasper adventures and extensions. 

Students and teachers in multiple classrooms at remote sites form a community of learners 
linked together by teleconferencing technology. These students participate in a specially prepared, 
interactive television show. During the show students' answers are relayed to our studios by 
phone and these data are then graphically presented to students so they can see how their answers 
compare to those of their peers in other cities and states. The challenges provide an opportunity for 
students and teachers to engage in formative self assessment. 

Over the past two years the Cognition and Technology Group at Vanderbilt has conducted 
three pilot Satellite Challenges that have been uplinked to over 30 sites across seven states 
(Cognition and Technology Group at Vanderbilt (CTGV), 1992a). Variations on the satellite 
events are continuing to be developed and studied. We eventually named these events SMART 
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Challenges, where SMART stands for "Special Multimedia Arenas for Refining Thinking." In this 
article we describe the evolution of the SMART challenges and discuss the cha'lenge based 
assessment model that has developed. 

BACKGROUND 

There have been two major forces behind the development of the SMART Challenges. One 
is the recognition of a need for learning communities in which students, teachers, school 
administrators and community members can engage in continual processes of inquiry and 
knowledge building. The other is a need for alternative assessment that can help to inSform and 
reform the curriculum as it is currently practised. Systemic assessments (Fredencksen & Collins, 
1989) can provide immediate and ongoing feedback to teachers and students, helping them to 
improve the quality of their teaching and learning within a complex problem solving curriculum. 
They can also be used to help collect data that can be analyzed and function as a means to evaluate 
the program and its effectiveness. The first SMART Challenge emerged as a response to our 
experience implementing the Jasper series in nine states (CTGV, in press). Since then the 
Challenges have been piloted three times and refined as a result of our research experiences. 

SMART CHALLENGES 

Leading to SMART Challenge I 

The importance of the community of learners model was recognized in the design of the 
nine state Jasper implementation project conducted in 1990-91 and use of this model was found to 
be increasingly more important as the project progressed. In the planning of the implementation 
project it seemed clear to us that the teachers who were going to teach Jasper neeued to feel part of 
a community that was larger than themselves. This was why we had attempted to have a minimum 
of two Jasper teachers per school, and why we hatf paired each Jasper site with someone from a 
local corporation who could offer technical and motivational support. 

Based on the research literature on the development of skilled performance and expertise 
(e.g. Chase & Simon, 1972), it also seemed clear that even the two weeks of intensive 
professional development that we were able to offer our teachers and corporate representatives 
would need to be supplemented with ongoing opjx>rtunities for learning. We assumed that our 
corporate representatives would be particularly helpful in answering questions about the 
technology the teachers were using, but we also anticipated questions about pedagogy. This was 
why we provided all our teachers with an electronic account to American Online. Our thoughi was 
that teachers would use this to contact our center; we had people on staff whose job was to monitor 
their messages and respond. As we discuss below, there were reasons why this resource was 
underutilized. 

As the first year of our Jasper implementation unfolded, we began to discover additional 
reasons for focusing attention on a learning communities model. One involved the importance of 
support from within the school. Nearly every one of cur teachers commented that their ability to 
implement the new ideas associated with the Jasper curriculum was affected strongly by the 
strength of support from their school principal. There is a research literature that emphasizes the 
importance of support from the principal (e.g. Hallinger & Murphy, 1993; Murphy, 1991;) but we 
had been only dimly aware of it at that time. Therefore, we did not systematically help teachers 
focus on plans for developing school wide and community support. 

Luckily, many of our teachers implemented their own plans for developing support from 
school and community members. For example, they invented ways to help parents and other 
community members understand the value of Jasper and the experiences related to it Several of our 
teachers invited parents and other adults to solve a Jasper adventure, and they used the students as 
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"experts" to keep the adults from getting too far off track. This idea proved to be a very powerful 
approach to community building. We have used it many times in our own community— always with 
extremely favorable results. 

An additional positive outcome that often accompanies opportunities for community 
members to solve the Jasper problems is that they often volunteer to help students learn more 
about topics featured in the Jasper adventures-topics such a boats, airplanes, principles of flight, 
eagles and other endangered species, recycling, and ideas for viable business plans. And teachers 
often got community members to help when students created and implemented their own projects 
such as their own trips to interesting locations, funded by their own business plans. 

Another reason for building learning communities emerged several months into the year, 
when we saw that only a subset of the teachers were making regular use of America Online to 
communicate with us and with one another. A major reason (which was obvious in retrospect but 
not in prospect) was that they had almost no free time during school hours for doing this, and since 
they did not have computers and modems at home they could not communicate from there. In 
addition, there was usually no compelling reason for them to communicate with one another and 
with us. We began to think about the possibility of developing high stakes events that would 
stimulate a greater exchange of ideas and information. 

In addition to our observations of how busy our teachers were during the school day, we 
noticed how many types of local events took precedence over Jasper-related activities, even though 
teachers and students wanted more time to work on Jasper. Events such as state mandated testing 
represent cases in point. We began to see that attempts to change (and hopefully improve) the 
nature of teaching and learning in classrooms will always compete with other priorities and that a 
major factor for change might be to create new priorities that are viewed by the community as even 
more important than the competing ones. Again we were led to including high stakes events within 
the Jasper curriculum. 

Finally, we began to see the importance of mechanisms for creating a dynamic curriculum. 
For example, individual teachers frequently generated new ideas for teaching Jasper concepts and 
for linking them to other areas of knowledge; we wanted to be able to communicate these ideas to 
other teachers. At the same time, our observational and interview studies (e.g. CTGV, in press; 
Goldman et al, 1991) were helping us find ways to improve the learning of students. For example, 
by using "what if questions after solving particular Jasper adventures, we were able to help 
students deepen their understanding and make transfer more flexible (e.g., CTGV, 1993). 
Similarly, we discovered ways of constructing visual representations of situations that made 
especially difficult concepts easier to comprehend and communicate. And as we worked with local 
teachers to use these insights to improve the learning experiences of their students, we began to sec 
that middle school students could reach higher levels of performance than we had anticipated 
initially. We wanted to find ways to communicate these possibilities to other members of our 
collaborative team. So we began to consider design principles for building a learning community 
that was capable of continual evolution and change. 

Other lessons we learned from the nine slate Jasper implementation not only emphasized 
the importance of fostering learning communities, but they also pointed to a need for alternative 
forms of assessment to provide feedback to students, teachers, and outside evaluators, including 
our research team. 

During the year of the nine state implementation we received data from the sites about 
performance on our assessment instruments. There were clear indications of effective learning and 
transfer in all sites (see especially CTGV, 1992a; Pellcgrino et al, 1991). Nevertheless, we also 
noted that many students' scores were far from perfect and that there was often considerable 
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variability in performance across different schools and classes within schools-even in cases where 
pretest scores on standardized math achievement tests would suggest that students should perform 
relatively similarly. This led to the conjecture that different teachers were probably teaching the 
Jasper adventures differently and that this affected the quality of their students' learning. 
Observations of classrooms within our local area reinforced this belief. So we began to think 
about ways to help the teachers, as well as our research team, continually re-assess the quality of 
teaching as well as the standards of achievement that were reasonable for students. In many 
classes, we felt that the standards were too low. 

From the nine state project we also discovered that teachers and especially students had 
grown to hate the paper-and-pencil assessment instruments that we had developed and 
administered (e.g., CTGV, 1992b). This prompted us to think about ways to preserve the value of 
assessment for learning (especially formative assessment) without making the students and 
teachere feel "tested." The metaphor we preferred was to give students a sense of being mentored 
by people who were on their side and wanted to help them achieve remarkable levels of 
performance. Cleariy new models of assessment were needed. 

Together these experiences led to our first design of a challenge based assessment model, 
which incorporated the development of a community of learners located in several states. This 
challenge based assessment experiment, SMART Challenge I, made use of teleconferencing 
technology. 

SMART Challenge I: Pick the Expert 

Our initial SMART Challenge involved one special event, a satellite teleconference focused 
around a game show called "Pick the Expert." Over 500 fifth and sixth grade students in 7 states 
prepared for and participated in the Challenge through the following stages: 

1. Complete an adventure. 

Students began to prepare for the "Pick the Expert" challenge by working with the Jasper 
adventure "Rescue at Boone's Meadow". In this adventure, Jasper finds a wounded eagle while 
on a fishing trip in the wilderness. By radio, he is able to communicate his location. Students help 
Emily, one of the characters in the video, as she plans to rescue the eagle. Solving this adventure, 
the students must evaluate a number of options such as the use of a car, an ultralight, or walking, 
while a considering numerous constraints such as speed of travel, distance, weight of cargo, and 
so forth. There are a number of possible options for saving the eagle; students discuss the 
strengths and weaknesses of each plan that they generate. 

2. Solve relevant analog and extension problems. 

After solving the rescue problem and discussing alternative plans, the students further prepared for 
the Jasper Challenge by solving related video-based analog and extension problems that engaged 
students in "what if* thinking such as: (a) What if the gas tank on the ultralight held 4 rather than 5 
gallons?; (b) What if the ultralight faced a 4 mile per hour headwind on this (shown on video) part 
of its flight? Students also worked with extension problems such as considering the issues that 
Charles Lindbergh had to consider when planning his flight from Paris to New York (he would 
have run cut of fuel had he not had a tailwind of at least 10 miles per hour). 

3. Participate in a Teleconference Based Uplink Event. 

The SMART Challenge involved a game show, "Pick the Expert," that featured three college 
students as contestants. Each claimed to be an expert on flight and on the Jasper adventure "Rescue 
at Boone's Meadow." Each "expert" was asked a series of questions about flight, including 
information about the effects of headwinds and tailwinds and whether, on a round-trip flight, a 10 
mph headwind (on the way) and tailwind (on the way back) would cancel one another out (they 
wouldn't). Students knew that their task was to listen to the experts' answers and judge for 
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themselves who was the true expert. A representative from each class would call in their votes 
during the live teleconference. 

The answers given by the expert contestants were scripted. All contestants were correct on 
the first round of questioning. By the fourth round everyone except the true expert had made some 
erroneous arguments. It was at this point in the teleconference that the students were to call in their 
votes. 

Reactions to SMART Challenge I 

The teleconference turned out to be an extremely popular event for our Jasper sites. First, 
students enjoyed the preparation phase for the challenge. We knew from previous data that 
students liked solving "Rescue at Boone's Meadow" (stage 1, above). Our new data indicated 
that, in contrast to earlier reactions to our paper and pencil assessment instruments, students 
enjoyed working on the video-based analog and extension problems (stage 2, above). Teachers 
indicated that students felt they were learning something new (about headwinds and tailwinds) 
rather than simply going through the motion of practicing a specific set of "old" skills. 

On the day of the teleconference (stage 3, above), videotaped and oral reports from teachers 
revealed groups of students actively discussing the strengths and weaknesses of each contestant's 
answers. At voting time, 85% of the students picked the real expert. To do so required them to 
understand that a trip of 65 miles with a headwind, followed by a return trip of 65 miles with a 
tail wind, did not mean that the headwind and tailwinds canceled each other out. 

Lessons Learned from SMART Challenge I 

In addition to the enthusiasm of teachers and students, we learned two more lessons from 
our initial SMART challenge that seem particularly noteworthy. First, the event was indeed high 
stakes for teachers and schools, and that had several advantages. Cables that were supposed to be 
run to a school "sometime soon" were run in time for the Challenge; uplinks that were slated "for 
the near future" became available in time for the Challenge; school events that might otherwise have 
interfered with working on Jasper and its extensions became second priority to the Challenge. In 
general, the Challenge seemed to help a number of things get accomplished that otherwise might 
have slipped by the wayside. 

A second lesson SMART Challenge I was that it was less challenging than students had 
anticipated and wanted. A number of students wrote us that they associated Jasper with tough 
challenges, and that the Challenge had not been tough enough. Actually, the information that 
students had to understand (about the headwind and tailwind on a round trip not canceling each 
other out) was quite sophisticated. But it seemed easy to the students because uieir preparation 
activities were very close to the events that they had to judge on the game show. We had worried 
about too many students being wrong during our first Challenge, so we had created a close link 
between the preparation during phases 1 and 2 and the actual information needed for the Challenge. 
The students' comments suggested that we had erred on the side of making things too easy. 

SMART Challenge II: Rate that Plan 

Our second SMART Challenge also involved one special event, again a satellite 
teleconference. However, this time the challenge focused on a more difficult game show called 
"Rate that Plan." Students prepared for and participated in the Challenge through the following 
stages. 

1. Complete an adventure. 

Students began to prepare for "Rate that Plan" by working with the Jasper adventure "The Big 
Splash." In this adventure, Jasper's young friend, Chris, wants to help his school raise money to 
help buy a new camera for the school TV station. His idea is to have a dunking booth in which 
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teachers could be dunked when students accurately hit a target. He must develop a business plan 
for the school principal in order to obtain a loan for his project. The overall problem centers 
around developing this business plan including the use of a statistical survey to help him decide if 
his idea would be a money maker. The problem * r >sed at the end of the video is to prepare the 
business plan that Chris should present to the Principal. The problem can be approached from 
multiple perspectives and requires the evaluation of multiple elements and options to construct i .-a 
acceptable alternative that meets the constraints originally set by the principal. 

2. Solve relevant analog and extension problems. 

Students next worked on "what if extensions of "The Big Splash." For example, in 'The Big 
Splash", Chris sampled every sixth person in the lunch line. What if he had sampled every fifth 
person who entered the school on Monday morning; 1/2 of all the students in his homeroom; or 
70% of all the students at baseball practice? Similarly, Chris decided not to fill the dunking booth 
pool with water from a firetruck even though the water would be free because there was too much 
risk that the fire department would be called out on a fire. Students discussed the levels of risk that 
would be acceptable to them. The "what if* problems also included pitches by "hucksters" about 
their plans for the fun fair and why the plans were superior to Chris'. Many of the plans were 
flawed (e.g., some were based on nonrepresentative samples). Students learned to detect the flaws 
and explain how the hucksters* arguments could be improved. 

3. Participate in a Teleconference-Based Uplink Event. 

On the day of the teleconference students were asked to rate individual business plans on the basis 
of their effectiveness. These plans were proposed by contestants, each of whom was trying to sell 
a plan to the class as a way the class might work together to raise money for an imaginary school 
"Fun Fair." Embedded in the contestants* proposals were all the data students would need to 
assess the merits of an individual plan. Two of the three plans contained flaws such as the use of 
non-representative samples or inaccurate estimates of expenses. Students had 45 seconds to rate 
each plan and explain any flaws. They then had the opportunity to watch three college panelists 
who asked questions of the contestants and thereby provided additional information students could 
use for their ratings. Students were then given a second chance to rate each plan and explain what 
was good and bad. At the end of the show, students were asked to choose the plan they thought 
would be most effective for them at their school's imaginary Fun Fair. As in the previous 
challenge, students from all over the country called in their ratings for the best plan and were then 
shown the ratings of their peers, as well as the ratios of the college panelists who appeared on the 
show. Students also knew that they were to mAl in their answer sheets which asked them to 
explain the reasons for their ratings. 

Reactions to SMART Challenge II 

As was true of the first challenge, teachers and students were highly enthusiastic. During 
the show, teachers reported that the discussions following the uplink were spirited and were 
motivated by the opportunity students had to compare their answers to those of peers across the 
country and to the college panelists who appeared in the uplink. A number of the teachers felt that 
the discussions were especially helpful to students who had originally overlooked flaws in various 
plans because it helped them improve their understanding of key aspects of statistics (e.g. random 
sampling) that were necessary in order to create effective business plans. 

Some representative comments made by teachers on our survey forms include the 
following: 

The Cfiallenge broadcast is a much needed aspect of the project. It proved a great 
reinforcement of skills and allows many extensions to existing projects. 
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The Challenge series aids in the continuity oftlte whole Jasper concept. Students 
consider themselves " experts" after solving each Jasper episode. They need 
opportunities to extend and expand their newfound knowledge. Their skills need 
to be fine tuned so they (students) will continue to use these skills in everyday math 
situations. 

I think the children enjoyed the culminating event. 

Lessons Learned from SMART Challenge II 

As with the previous challenge, we observed that a high stakes event such as a SMART 
challenge made things happen that otherwise might have taken a long time to accomplish. We 
learned three new lessons as well. The first was that our format was now too hard because 
students did not have enough time to evaluate the strengths and weaknesses of each business plan. 
We had been aware of this possible difficulty when planning the challenge, but had little choice 
because we had only one 45 minute time slot when all our classes could participate in the 
challenge, and we could not afford to go on the air for a second time slot. We had hoped that our 
use of the college panel to ask questions about each plan, coupled with the opportunity to rate each 
plan twice, would overcome the severe time crunch imposed by the constraints of linear television. 
But students remarked quite ademantly that they wanted the time to actually work through each 
business plan. 

A second lesson we learned was that we needed information about students' reasons for 
choosing the best business plan rather than only the call-in-data we received about their choice for 
the best plan. As it turned out, the vast majority of the students choose a plan that was actually 
flawed— a plan to sell ice cream at the fun fair. Did this mean that they had not learned well while 
preparing for the challenge, or that their votes were based on something else? 

It was not until we received the students' answer sheets with reasons for choices, plus had 
the chance to interview some students, that we were able to decide whether they had learned or not. 
As it turned out, many classes had learned well whereas other had not. Those who seemed to have 
learned were ones who explained that (a) the plan they chose was flawed (it was based on data 
from a very biased sample of respondents), but (b) it would probably have been the best plan even 
if the sample had been representative, In contrast, other students seemed to have voted for the ice 
cream plan without realizing that it was based on biased data. 

We tried to understand why a number of students had not noticed flaws in several of the 
business plan. Was this because the concepts were too difficult for them or because the teachers 
had not prepared them for the challenge, (either because of a lack of time, lack of interest, or 
perhaps because of an inadequate understanding of the concepts required for the preparation)? In 
order to answer these questions, we decided that the next step in our research needed to be to keep 
a better eye on what happened in the classroom. This became the goal of our next SMART 
Challenges, IIIA & IIIB. 

SMART Challenge III A: Local Presentations 

In our third attempt to explore SMART challenges we decided to work with only a single 
local classroom so that we could carefully observe the teaching and learning sessions prior to the 
challenge and see how they related to the performance of students during the challenge itself. We 
upped the stakes of the challenge and the preparation for it, while dropping the teleconference 
format this time. After solving "The Big Splash" and its analogs and extensions, students were to 
work in groups to invent their own business plans and collect data to support their arguments about 
the viability of the plans (i.e., to estimate projected income and profit). For the culminating event 
each group of students would present its own business plan to a panel of experts and answer 
questions asked by these experts. These experts included local business, university, and 

page 7 - Originally presented at AERA, Atlanta on April 15, 1993 



ERLC 



9 



educational leaders. If the plans were sound, the school principal would lend each group the 
money to actually carry out its plan. 

The teacher with whom we worked on this project was an experienced Jasper teacher who 
has become an invaluable colleague. We observed his classroom daily for the 8 weeks of 
instruction that it took to complete this unit, and we worked with him to introduce some new 
instructional tools that we felt would facilitate student learning, including self-assessment devices, 
(reference this chapter) At the end of each day we were able to discuss ideas with the teacher 
through a computer-based two-way videoconferencing system that connected his classroom with 
theLTC. 1 

We were able to observe the effects of the students' self-assessment on classroom 
discussion and also the nature of classroom discussions about key mathematical concepts such as 
representative sampling, optimal sample sizes and so forth. In addition, we encouraged the teacher 
to give students multiple experiences for making presentations and we videotaped the progress of 
the students as they made their presentations-first about the income portion of the solution to "The 
Big Splash", then about their thoughts on expenses, later about their thought on the entire plan 
(income, expenses and degree of anticipated profit). Later presentations were about each group of 
students' own business plans and the data gathered to assess its feasibility. These were all in 
preparation for the culminating event, the presentations of the students' own business plans to the 
panel of community leaders. 

Reactions to SMART Challenge III A 

Our experienced teacher was extremely pleased by his students' progress. He had taught 
Jasper for three years and felt that, without a doubt, his students had reached levels of 
understanding and skill (including presentation skill) that he never before imagined possible for 
students of this age. Our panel of experts who interviewed the students about their own ideas for 
business plans was also extremely impressed by the high level of performance of the groups- 
including their abilities to think on their feet while answering questions asked by panel members. 

Lessons Learned from SMART Challenge III A 

Our classroom observations convinced us of the importance of helping teachers make the 
thoughts, feelings and skills of their students visible. One way that helped accomplish this was a 
simple device that provided frequent opportunities for anonymous self-assessment by the students. 
Our teacher felt that these assessments provided an excellent opportunity for him to better 
understand what the class was thinking or feeling and to then take appropriate action. 

Probably the most important lesson learned from our classroom observations involved 
issues of professional development. A number of issues arose during class discussions that 
pushed all of us-the teacher plus members of our research team-to the edges of our knowledge. 
For example, what was a way to clarify the reasoning underlying the mathematical procedure used 
to estimate the best ticket price in "The Big Splash"? At a more general level, what are the general 
principles for deciding on sample size? This became important for students developing their own 
business plans. If their school population size was twice as large as the one in the Jasper 
adventure (Chris's population size was 360), should the sample size be 120, remain at 60 or what? 
Many members of our research group were not sure about the answers to these questions, and the 
same is true of teachers because they did not have the benefit of in-depth preparation in statistics 
when they attended school. In addition, most teachers with whom we have worked are not 
familiar with how visual diagrams can be used to help students understand complex concepts- 
diagrams that members of our research team found themselves using in order to communicate with 
one another. 

These experiences helped us realize how much we as a research team rely on one another to 
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continually learn new information that we need to know, and how difficult it is in most classrooms 
for teachers to have similar advantages. So we began to think about another important function for 
our SMART Challenges: To provide new information to teachers as well as students, and to 
structure situations so that we as a research team could also learn from the teachers and students. 
These goals shaped our work for SMART Challenge IV, which is discussed below. First, 
however, we discuss SMART Challenge III B. 

SMART Challenge III B: Mentoring with College Students 

This challenge was a variant of SMART Challenge III A and involved a new class of sixth 
grade students. The major difference in Challenge III B was the addition of a "two way 
mentoring" program involving the thirty sixth graders and thirty college students who were taking 
a class in "Cognition and Instruction" at Peabody College at^Vanderbilt. Each class was divided 
into 6 groups of students, and each pair of groups (one from the sixth grade and one from the 
Peabody class) first met face - to - face. Subsequent interactions were carried out via computer- 
based two-way videoconferencing facilities that connected Peabody College with the sixth grade 
classroom. Each group of sixth graders and their respective college counterparts "met" for half an 
hour once a week to discuss students' progress as they proceeded from solving "The Big Splash" 
to creating and presenting their own business plans. 
Reactions to SMART Challenge III B 

SMART Challenge III B is still ongoing as we write this paper. Nevertheless, we have 
already observed a number of reactions to it that seem very clear. 

First, the experience for the college students has been valuable. It provided them with the 
opportunity to see a number of the issues discussed in class actually exemplified in the real world. 
For example, several of the college student groups were worried that their sixth grade students 
seemed to take no initiative in thinking through issues such as how to begin problem solving or to 
present their ideas in a coherent manner. Instead, they simply wanted the college students to tell 
them the answers. Experiences such as these brought to life a number of class readings and 
discussions that focused on the fact that many students have not learned to think for themselves in 
a wide variety of domains.(e.g., see Bransford, Goldman & Vye, 1991, Nickerson, 1988; 
Resnick, 1987). When the college students encountered these problems in the sixth graders, the 
issue of teaching thinking they became much more real to them. It was then possible to help the 
college students devise strategies that would help make the sixth graders more generative in their 
activities and to relate this to course readings and discussions. Overall, the introduction of the 
mentoring activities had an extremely positive effect on the college class. 

Second, the sixth graders learned a great deal from the mentoring experiment. The teacher 
indicated that they looked forwarded to meeting with their college group and viewed them as 
mentors rather than as "testers". The sixth graders also began to discover that the college students 
were human; many of the sixth graders seemed to begin the project with the misconception that 
"smart" college students would have no trouble solving a problem like "The Big Splash." It was 
very helpful to them to hear that the college students had solved the problem in their class and had 
found it very challenging. In fact, the majority of the college students did not come up with 
optimal solutions on their own. In addition, it was helpful to the sixth graders to learn that a 
number of the college students were also quite nervous about making public presentations to 
groups. 

Lessons Learned from SMART Challenge III B 

An important lesson is that a number of logistics issues must ultimately be solved to make 
mentoring activities run more smoothly. Interruptions to the schedules of the middle school 
students were frequent and often sudden. Unfortunately, quick communication with the college 
students was usually difficult. In addition, substitute teachers often did not feel comfortable with 
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the videoconferencing technology and did not want to use it. For projects of this kind, there is a 
ne^d to ensure that teachers, substitute teachers, college professors and college students are more 
easily able to communicate. We are working on alternative telecommunications facilities that should 
facilitate this process. 

Another important lesson is related to the "need for professional development" lesson 
learned in Challenge III A. Like many middle school teachers, most of the college students had not 
yet had courses in statistics and hence quickly found themselves on the edge of their knowledge 
when attempting to work with the sixth graders. This provided further motivation to increase our 
focus on the role of professional development in SMART challenges, as addressed in Challenge 
IV, below. 

SMART Challenge IV: Toward More Frequent Opportunities for Mutual Learning 

SMART Challenge IV is just beginning to be implemented as we write this paper. Based 
on lessons learned from the earlier SMART challenges, it is designed to assess the value added of 
increasing the opportunities for mutual learning by teachers, students and our research team. It 
culminates in a special live televised event called "The Big Challenge", which is a variant on 
SMART Challenge IF s Rate that Plan game show. It involves a number of experimental and 
control classrooms, which differ in their preparation stages. 

The control classrooms in SMART Challenge IV receive experiences similar to those 
discussed in SMART Challenge II, above. Students prepare for The Big Challenge by solving 
"The Big Splash", solving analog and extension problems relevant to it, and then spending several 
days generating their own business plans and deciding how they would collect data to estimate the 
projected income for their plans. 

Students and teachers in the experimental classrooms are also preparing for the live TV 
show The Big Challenge by solving "The Big Splash", solving analog and extension problems 
relevant to it, and then spending several days generating their own business plans and deciding 
how they would collect data to estimate the projected income for their plans. In addition, students 
and teachers in the experimental classrooms are provided with more frequent opportunities for 
mutual learning. This occurs by way of four special video-based programs that occur prior to The 
Big Challenge. The first program provides an introduction to the series; the purpose of the three 
remaining programs is to help students and teachers focus on specific sub-challenges while also 
providing opportunities to (a) compare their answers to those of students in other experimental 
classrooms throughout the city and (b) see models of powerful ways to express ideas. 

At the end of the preparation stages, both experimental and control classrooms participate in 
the live interactive "Big Challenge" show that allows them to test their mettle with respect to a 
number of issues relevant to business p'ans and "The Big Splash." Data we are collecting both 
during and after this challenge will help us assess the value-added of the video-based programs 
made available to classes in the experimental condition. We expect to find advantages for student 
understanding, for students' abilities to give clear presentations (including their use of graphics to 
communicate effectively) and for teachers' understanding of statistics and how they can help 
students to better understand statistical concepts. 

Future Plans 

Our hope is that SMART Challenge IV will provide us with information and experience that 
will allow us to refine our ideas during the summer of 1993 and prepare to implement key learning 
community ideas in a larger number of schools in the Nashville Metropolitan School System 
during the 1993- 1994 academic year. Our next SMART Challenge project will be extended to one 
year and will cover Jasper trip planning adventures as well as statistics and business planning 
adventures. The school system's educational TV studio has given us permission to use their 
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facilities to deliver SMART challenges throughout the school year. 

An extremely important issue is one of ensuring that students will not be hurt on state tests 
of accountability by spending so much time working on SMART challenges. Our current plan for 
dealing with this issue is to ask schools to agree to implement a year long, daily "project hour" that 
provides time for problem-based and problem or project-based activities such as Jasper without 
taking time away from ongoing instruction in mathematics, science, reading or other key activities. 
We then plan to gradually integrate the instruction in the mathematics classes by working backward 
from Jasper adventures to find relevant concepts and skills that students are covering in their math 
classes. And we plan to merge our Jasper-related instruction with a unique approach to skills-based 
assessment that has been developed by our colleagues Doug and Lynn Fuchs (1992). In this 
manner, we hope to show strong gains on standardized tests while also helping students develop 
the complex thinking and communication skills that the Jasper adventures are designed to afford. 
Furthermore, we plan to continue our work with two-way mentoring involving college students 
and extend the experience to other members of the community such as representatives of local 
businesses who want to make a contribution to the schools. 

Needless to say, our plans for next year are still only a small part of what one needs in 
order to develop an effective learning community. But we want to make sure that we do not attempt 
to do more than we can manage— we are right on the edge as it is. In subsequent years we hope to 
extend our work to younger and older grades and to other curriculum areas. And we plan to 
continue to collect data to determine what works and what needs to be changed. 

SUMMARY AND CONCLUSIONS 

The general model of challenge-based assessment that is being developed is applicable to 
any problem-based curriculum, and in its current version, consists of the following six stages. 

1. Complete a problem or project based unit of instruction (e.g., an episode of 
Jasper) 

2. Review solutions and practice additional problems to deepen and broaden 

understanding of concepts, procedures, and cc .nputations in preparation for a 
new challenge. 

3. Solve a new challenge problem similar to that of the original problem (stage 1). 
Work on class projects that are related to the theme being studied. 

4. Participate with students in many other classrooms in video based performance 
events related to stages 1, 2, and 3. These events showcase the best student 
solutions, presentations, and projects; provide expert models for students and 
teachers; and provide students with immediate feedback on their performance 
and that of their peers. 

5. Conclude with a culminating video-based event that is highly motivating for 
students and teachers. 

6. Repeat with a new unit of instruction. 

The impact of classroom-based technology on learning can be extended through the 
innovative use of distance learning technologies in conjunction with our theoretical model of 
challenge-based assessment. When high stakes, high visibility events like the "Big Challenge" or 
or the Jasper Challenge Series take place students and teachers in remote sites are supported by 
local school authorities and communities. However, we found that to ensure student preparation 
students and teachers need to be prepared in ways that go beyond a final high stakes event. 
Students and teachers are connected with one another across multiple classrooms to form a 
mediated community of learners that share goals performance and a common context. Our 
theoretical model for challenge-based assessment, as exemplified by the Jasper Challenge Series, is 
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an example of how formative assessment can be used to highly motivate students and teachers and 
drive instruction. This is especially important for problem and project based curriculums which 
require alternatives to traditional assessment instruments. It is our hope that in the future we will 
be able to extend our theoretical model across the curriculum to include other domains in science, 
language arts and social studies. 
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FOOTNOTE 

1 We would like to gratefully acknowledge the assistance of Northern Telecom and their generosity 
in allowing us to use their ** Visit" ISDN based desk top teleconferencing software in our 
cortinuing research work. 
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